- Title
- Model compression for IoT applications in industry 4.0 via multiscale knowledge transfer
- Creator
- Fu, Shipeng; Li, Zhen; Liu, Kai; Din, Sadia; Imran, Muhammad; Yang, Xiaomin
- Date
- 2020
- Type
- Text; Journal article
- Identifier
- http://researchonline.federation.edu.au/vital/access/HandleResolver/1959.17/184611
- Identifier
- vital:16536
- Identifier
-
https://doi.org/10.1109/TII.2019.2953106
- Identifier
- ISBN:1551-3203 (ISSN)
- Abstract
- Recently, Industry 4.0 has attracted much attention. It has close relations with the Internet of Things (IoT). On the other hand, convolutional neural networks (CNNs) have shown promising performance in many foundational services of the IoT applications. For the IoT applications with high-speed data streams and the requirement of time-sensitive actions, fast processing is demanded on small-scale platforms or even on IoT devices themselves. Therefore, it is inappropriate to employ cumbersome CNNs in IoT applications, making the study of model compression necessary. In knowledge transfer, it is common to employ a deep, well-trained network, called teacher, to guide a shallow, untrained network, called student, to have better performance. Previous works have made many attempts to transfer single-scale knowledge from teacher to student, leading to degradation of generalization ability. In this article, we introduce multiscale representations to knowledge transfer, which facilitates the generalization ability of student. We divide student and teacher into several stages. Student learns from multiscale knowledge provided by teacher at the end of each stage. Extensive experiments demonstrate the effectiveness of our proposed method both on image classification and on single image super-resolution. The huge performance gap between student and teacher is significantly narrowed down by our proposed method, making student suitable for IoT applications. © 2005-2012 IEEE.
- Publisher
- IEEE Computer Society
- Relation
- IEEE Transactions on Industrial Informatics Vol. 16, no. 9 (2020), p. 6013-6022
- Rights
- All metadata describing materials held in, or linked to, the repository is freely available under a CC0 licence
- Rights
- Copyright @ IEEE
- Subject
- 40 Engineering; 46 Information and Computing Sciences; Image classification; Industry 4.0; Internet of Things (IoT); Knowledge transfer; Multiscale representations; Single image super-resolution
- Reviewed
- Funder
- This work was supported in part by the National Natural Science Foundation of China under Grant 61701327 and Grant 61711540303 and in part by the Science Foundation of Sichuan Science and Technology Department under Grant 2018GZ0178.
- Hits: 668
- Visitors: 388
- Downloads: 0
| Thumbnail | File | Description | Size | Format |
|---|